Prying the Lid from the Black Box: Plotting Evaluation Strategy for Welfare Employment and Training Programs

To date, most evaluations of welfare-related employment and training (WRET) programs have focused on the difference between outcomes in a single site for people who receive the program's "treatment" and those in a control group who do not. This paper argues that progress in determining what makes programs effective requires greater emphasis on planning for synthesis of results from multiple sites and perhaps multiple treatment variations to provide insight into the program's production function: the relationship among outcomes and the characteristics of the program tested, the environment of implementation, and the people who participated. The authors develop a multilevel model of WRET outcomes and use the model (1) to determine analytically the number of sites and observations per site required to achieve a target level of efficiency in estimating production function parameters, (2) to structure a review of recent multisite WRET program evaluations, and (3) as a foundation for a suggested multisite program evaluation agenda. Prying the Lid from the Black Box: Plotting Evaluation Strategy for Welfare Employment and Training Programs Table of